{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SQL Join Query Engine\n",
    "In this tutorial, we show you how to use our SQLJoinQueryEngine.\n",
    "\n",
    "This query engine allows you to combine insights from your structured tables with your unstructured data.\n",
    "It first decides whether to query your structured tables for insights.\n",
    "Once it does, it can then infer a corresponding query to the vector store in order to fetch corresponding documents."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# NOTE: This is ONLY necessary in jupyter notebook.\n",
    "# Details: Jupyter runs an event-loop behind the scenes.\n",
    "#          This results in nested event-loops when we start an event-loop to make async queries.\n",
    "#          This is normally not allowed, we use nest_asyncio to allow it for convenience.\n",
    "import nest_asyncio\n",
    "\n",
    "nest_asyncio.apply()\n",
    "\n",
    "import logging\n",
    "import sys\n",
    "\n",
    "logging.basicConfig(stream=sys.stdout, level=logging.INFO)\n",
    "logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index import (\n",
    "    VectorStoreIndex,\n",
    "    SimpleDirectoryReader,\n",
    "    ServiceContext,\n",
    "    StorageContext,\n",
    "    SQLDatabase,\n",
    "    WikipediaReader,\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create Common Objects\n",
    "\n",
    "This includes a `ServiceContext` object containing abstractions such as the LLM and chunk size.\n",
    "This also includes a `StorageContext` object containing our vector store abstractions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# # define pinecone index\n",
    "# import pinecone\n",
    "# import os\n",
    "\n",
    "# api_key = os.environ['PINECONE_API_KEY']\n",
    "# pinecone.init(api_key=api_key, environment=\"us-west1-gcp\")\n",
    "\n",
    "# # dimensions are for text-embedding-ada-002\n",
    "# # pinecone.create_index(\"quickstart\", dimension=1536, metric=\"euclidean\", pod_type=\"p1\")\n",
    "# pinecone_index = pinecone.Index(\"quickstart\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# # OPTIONAL: delete all\n",
    "# pinecone_index.delete(deleteAll=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index.node_parser.simple import SimpleNodeParser\n",
    "from llama_index import ServiceContext, LLMPredictor\n",
    "from llama_index.storage import StorageContext\n",
    "from llama_index.vector_stores import PineconeVectorStore\n",
    "from llama_index.langchain_helpers.text_splitter import TokenTextSplitter\n",
    "from llama_index.llms import OpenAI\n",
    "\n",
    "# define node parser and LLM\n",
    "chunk_size = 1024\n",
    "llm = OpenAI(temperature=0, model=\"gpt-4\", streaming=True)\n",
    "service_context = ServiceContext.from_defaults(chunk_size=chunk_size, llm=llm)\n",
    "text_splitter = TokenTextSplitter(chunk_size=chunk_size)\n",
    "node_parser = SimpleNodeParser(text_splitter=text_splitter)\n",
    "\n",
    "# # define pinecone vector index\n",
    "# vector_store = PineconeVectorStore(pinecone_index=pinecone_index, namespace='wiki_cities')\n",
    "# storage_context = StorageContext.from_defaults(vector_store=vector_store)\n",
    "# vector_index = VectorStoreIndex([], storage_context=storage_context)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create Database Schema + Test Data\n",
    "\n",
    "Here we introduce a toy scenario where there are 100 tables (too big to fit into the prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sqlalchemy import (\n",
    "    create_engine,\n",
    "    MetaData,\n",
    "    Table,\n",
    "    Column,\n",
    "    String,\n",
    "    Integer,\n",
    "    select,\n",
    "    column,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "engine = create_engine(\"sqlite:///:memory:\", future=True)\n",
    "metadata_obj = MetaData()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# create city SQL table\n",
    "table_name = \"city_stats\"\n",
    "city_stats_table = Table(\n",
    "    table_name,\n",
    "    metadata_obj,\n",
    "    Column(\"city_name\", String(16), primary_key=True),\n",
    "    Column(\"population\", Integer),\n",
    "    Column(\"country\", String(16), nullable=False),\n",
    ")\n",
    "\n",
    "metadata_obj.create_all(engine)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['city_stats'])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# print tables\n",
    "metadata_obj.tables.keys()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We introduce some test data into the `city_stats` table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sqlalchemy import insert\n",
    "\n",
    "rows = [\n",
    "    {\"city_name\": \"Toronto\", \"population\": 2930000, \"country\": \"Canada\"},\n",
    "    {\"city_name\": \"Tokyo\", \"population\": 13960000, \"country\": \"Japan\"},\n",
    "    {\"city_name\": \"Berlin\", \"population\": 3645000, \"country\": \"Germany\"},\n",
    "]\n",
    "for row in rows:\n",
    "    stmt = insert(city_stats_table).values(**row)\n",
    "    with engine.connect() as connection:\n",
    "        cursor = connection.execute(stmt)\n",
    "        connection.commit()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[('Toronto', 2930000, 'Canada'), ('Tokyo', 13960000, 'Japan'), ('Berlin', 3645000, 'Germany')]\n"
     ]
    }
   ],
   "source": [
    "with engine.connect() as connection:\n",
    "    cursor = connection.exec_driver_sql(\"SELECT * FROM city_stats\")\n",
    "    print(cursor.fetchall())"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load Data\n",
    "\n",
    "We first show how to convert a Document into a set of Nodes, and insert into a DocumentStore."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: wikipedia in /Users/jerryliu/Programming/gpt_index/.venv/lib/python3.10/site-packages (1.4.0)\n",
      "Requirement already satisfied: beautifulsoup4 in /Users/jerryliu/Programming/gpt_index/.venv/lib/python3.10/site-packages (from wikipedia) (4.12.2)\n",
      "Requirement already satisfied: requests<3.0.0,>=2.0.0 in /Users/jerryliu/Programming/gpt_index/.venv/lib/python3.10/site-packages (from wikipedia) (2.28.2)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /Users/jerryliu/Programming/gpt_index/.venv/lib/python3.10/site-packages (from requests<3.0.0,>=2.0.0->wikipedia) (2022.12.7)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /Users/jerryliu/Programming/gpt_index/.venv/lib/python3.10/site-packages (from requests<3.0.0,>=2.0.0->wikipedia) (3.1.0)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /Users/jerryliu/Programming/gpt_index/.venv/lib/python3.10/site-packages (from requests<3.0.0,>=2.0.0->wikipedia) (3.4)\n",
      "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /Users/jerryliu/Programming/gpt_index/.venv/lib/python3.10/site-packages (from requests<3.0.0,>=2.0.0->wikipedia) (1.26.15)\n",
      "Requirement already satisfied: soupsieve>1.2 in /Users/jerryliu/Programming/gpt_index/.venv/lib/python3.10/site-packages (from beautifulsoup4->wikipedia) (2.4.1)\n",
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip available: \u001b[0m\u001b[31;49m22.3.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.1.2\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "# install wikipedia python package\n",
    "!pip install wikipedia"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "cities = [\"Toronto\", \"Berlin\", \"Tokyo\"]\n",
    "wiki_docs = WikipediaReader().load_data(pages=cities)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build SQL Index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sql_database = SQLDatabase(engine, include_tables=[\"city_stats\"])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build Vector Index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total LLM token usage: 0 tokens\n",
      "> [build_index_from_nodes] Total LLM token usage: 0 tokens\n",
      "> [build_index_from_nodes] Total LLM token usage: 0 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total embedding token usage: 17064 tokens\n",
      "> [build_index_from_nodes] Total embedding token usage: 17064 tokens\n",
      "> [build_index_from_nodes] Total embedding token usage: 17064 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total LLM token usage: 0 tokens\n",
      "> [build_index_from_nodes] Total LLM token usage: 0 tokens\n",
      "> [build_index_from_nodes] Total LLM token usage: 0 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total embedding token usage: 17943 tokens\n",
      "> [build_index_from_nodes] Total embedding token usage: 17943 tokens\n",
      "> [build_index_from_nodes] Total embedding token usage: 17943 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total LLM token usage: 0 tokens\n",
      "> [build_index_from_nodes] Total LLM token usage: 0 tokens\n",
      "> [build_index_from_nodes] Total LLM token usage: 0 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total embedding token usage: 10600 tokens\n",
      "> [build_index_from_nodes] Total embedding token usage: 10600 tokens\n",
      "> [build_index_from_nodes] Total embedding token usage: 10600 tokens\n"
     ]
    }
   ],
   "source": [
    "# Insert documents into vector index\n",
    "# Each document has metadata of the city attached\n",
    "\n",
    "vector_indices = {}\n",
    "vector_query_engines = {}\n",
    "\n",
    "for city, wiki_doc in zip(cities, wiki_docs):\n",
    "    vector_index = VectorStoreIndex.from_documents([wiki_doc])\n",
    "    query_engine = vector_index.as_query_engine(similarity_top_k=2)\n",
    "    vector_indices[city] = vector_index\n",
    "    vector_query_engines[city] = query_engine"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define Query Engines, Set as Tools"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index.query_engine import SQLJoinQueryEngine, RetrieverQueryEngine\n",
    "from llama_index.tools.query_engine import QueryEngineTool\n",
    "from llama_index.tools import ToolMetadata\n",
    "from llama_index.indices.vector_store import VectorIndexAutoRetriever\n",
    "from llama_index.query_engine import SubQuestionQueryEngine\n",
    "from llama_index.indices.struct_store.sql_query import NLSQLTableQueryEngine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "query_engine = NLSQLTableQueryEngine(\n",
    "    sql_database=sql_database,\n",
    "    tables=[\"city_stats\"],\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index.query_engine import SubQuestionQueryEngine\n",
    "\n",
    "query_engine_tools = []\n",
    "for city in cities:\n",
    "    query_engine = vector_query_engines[city]\n",
    "\n",
    "    query_engine_tool = QueryEngineTool(\n",
    "        query_engine=query_engine,\n",
    "        metadata=ToolMetadata(\n",
    "            name=city, description=f\"Provides information about {city}\"\n",
    "        ),\n",
    "    )\n",
    "    query_engine_tools.append(query_engine_tool)\n",
    "\n",
    "\n",
    "s_engine = SubQuestionQueryEngine.from_defaults(query_engine_tools=query_engine_tools)\n",
    "\n",
    "# from llama_index.indices.vector_store.retrievers import VectorIndexAutoRetriever\n",
    "# from llama_index.vector_stores.types import MetadataInfo, VectorStoreInfo\n",
    "# from llama_index.query_engine.retriever_query_engine import RetrieverQueryEngine\n",
    "\n",
    "\n",
    "# vector_store_info = VectorStoreInfo(\n",
    "#     content_info='articles about different cities',\n",
    "#     metadata_info=[\n",
    "#         MetadataInfo(\n",
    "#             name='title',\n",
    "#             type='str',\n",
    "#             description='The name of the city'),\n",
    "#     ]\n",
    "# )\n",
    "# vector_auto_retriever = VectorIndexAutoRetriever(vector_index, vector_store_info=vector_store_info)\n",
    "\n",
    "# retriever_query_engine = RetrieverQueryEngine.from_args(\n",
    "#     vector_auto_retriever, service_context=service_context\n",
    "# )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "sql_tool = QueryEngineTool.from_defaults(\n",
    "    query_engine=sql_query_engine,\n",
    "    description=(\n",
    "        \"Useful for translating a natural language query into a SQL query over a table containing: \"\n",
    "        \"city_stats, containing the population/country of each city\"\n",
    "    ),\n",
    ")\n",
    "s_engine_tool = QueryEngineTool.from_defaults(\n",
    "    query_engine=s_engine,\n",
    "    description=f\"Useful for answering semantic questions about different cities\",\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define SQLJoinQueryEngine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "query_engine = SQLJoinQueryEngine(\n",
    "    sql_tool, s_engine_tool, service_context=service_context\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[36;1m\u001b[1;3mQuerying SQL database: Useful for translating a natural language query into a SQL query over a table containing city_stats, containing the population/country of each city\n",
      "\u001b[0mINFO:llama_index.query_engine.sql_join_query_engine:> Querying SQL database: Useful for translating a natural language query into a SQL query over a table containing city_stats, containing the population/country of each city\n",
      "> Querying SQL database: Useful for translating a natural language query into a SQL query over a table containing city_stats, containing the population/country of each city\n",
      "> Querying SQL database: Useful for translating a natural language query into a SQL query over a table containing city_stats, containing the population/country of each city\n",
      "INFO:llama_index.indices.struct_store.sql_query:> Table desc str: Schema of table city_stats:\n",
      "Table 'city_stats' has columns: city_name (VARCHAR(16)), population (INTEGER), country (VARCHAR(16)) and foreign keys: .\n",
      "\n",
      "> Table desc str: Schema of table city_stats:\n",
      "Table 'city_stats' has columns: city_name (VARCHAR(16)), population (INTEGER), country (VARCHAR(16)) and foreign keys: .\n",
      "\n",
      "> Table desc str: Schema of table city_stats:\n",
      "Table 'city_stats' has columns: city_name (VARCHAR(16)), population (INTEGER), country (VARCHAR(16)) and foreign keys: .\n",
      "\n",
      "INFO:llama_index.token_counter.token_counter:> [query] Total LLM token usage: 408 tokens\n",
      "> [query] Total LLM token usage: 408 tokens\n",
      "> [query] Total LLM token usage: 408 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [query] Total embedding token usage: 0 tokens\n",
      "> [query] Total embedding token usage: 0 tokens\n",
      "> [query] Total embedding token usage: 0 tokens\n",
      "\u001b[33;1m\u001b[1;3mSQL query: SELECT city_name, population FROM city_stats ORDER BY population DESC LIMIT 1;\n",
      "\u001b[0m\u001b[33;1m\u001b[1;3mSQL response: \n",
      "Tokyo is the city with the highest population, with 13.96 million people. It is a vibrant city with a rich culture and a wide variety of art forms. From traditional Japanese art such as calligraphy and woodblock prints to modern art galleries and museums, Tokyo has something for everyone. There are also many festivals and events throughout the year that celebrate the city's culture and art.\n",
      "\u001b[0m\u001b[36;1m\u001b[1;3mTransformed query given SQL response: What are some popular festivals and events in Tokyo that celebrate its culture and art?\n",
      "\u001b[0mINFO:llama_index.query_engine.sql_join_query_engine:> Transformed query given SQL response: What are some popular festivals and events in Tokyo that celebrate its culture and art?\n",
      "> Transformed query given SQL response: What are some popular festivals and events in Tokyo that celebrate its culture and art?\n",
      "> Transformed query given SQL response: What are some popular festivals and events in Tokyo that celebrate its culture and art?\n",
      "Generated 4 sub questions.\n",
      "\u001b[36;1m\u001b[1;3m[Tokyo] Q: What are some popular festivals in Tokyo\n",
      "\u001b[0mINFO:llama_index.token_counter.token_counter:> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [retrieve] Total embedding token usage: 7 tokens\n",
      "> [retrieve] Total embedding token usage: 7 tokens\n",
      "> [retrieve] Total embedding token usage: 7 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [get_response] Total LLM token usage: 1826 tokens\n",
      "> [get_response] Total LLM token usage: 1826 tokens\n",
      "> [get_response] Total LLM token usage: 1826 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [get_response] Total embedding token usage: 0 tokens\n",
      "> [get_response] Total embedding token usage: 0 tokens\n",
      "> [get_response] Total embedding token usage: 0 tokens\n",
      "\u001b[36;1m\u001b[1;3m[Tokyo] A: \n",
      "Some popular festivals in Tokyo include the Sannō at Hie Shrine, the Sanja at Asakusa Shrine, the biennial Kanda Festivals, the annual fireworks display over the Sumida River, picnics under the cherry blossoms in Ueno Park, Inokashira Park, and the Shinjuku Gyoen National Garden, and the Harajuku fashion and cosplay events.\n",
      "\u001b[0m\u001b[33;1m\u001b[1;3m[Tokyo] Q: What are some popular events in Tokyo\n",
      "\u001b[0mINFO:llama_index.token_counter.token_counter:> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [retrieve] Total embedding token usage: 7 tokens\n",
      "> [retrieve] Total embedding token usage: 7 tokens\n",
      "> [retrieve] Total embedding token usage: 7 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [get_response] Total LLM token usage: 1879 tokens\n",
      "> [get_response] Total LLM token usage: 1879 tokens\n",
      "> [get_response] Total LLM token usage: 1879 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [get_response] Total embedding token usage: 0 tokens\n",
      "> [get_response] Total embedding token usage: 0 tokens\n",
      "> [get_response] Total embedding token usage: 0 tokens\n",
      "\u001b[33;1m\u001b[1;3m[Tokyo] A: \n",
      "Some popular events in Tokyo include the Sannō at Hie Shrine, the Sanja at Asakusa Shrine, the biennial Kanda Festivals, an enormous fireworks display over the Sumida River, picnics under the cherry blossoms in Ueno Park, Inokashira Park, and the Shinjuku Gyoen National Garden, Harajuku's youth style, fashion and cosplay, and the Tokyo Marathon. Tokyo also hosts national and international sporting events such as basketball tournaments, women's volleyball tournaments, tennis tournaments, swim meets, marathons, rugby union and sevens rugby games, soccer exhibition games, judo, and karate.\n",
      "\u001b[0m\u001b[38;5;200m\u001b[1;3m[Tokyo] Q: What festivals in Tokyo celebrate its culture\n",
      "\u001b[0mINFO:llama_index.token_counter.token_counter:> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [retrieve] Total embedding token usage: 7 tokens\n",
      "> [retrieve] Total embedding token usage: 7 tokens\n",
      "> [retrieve] Total embedding token usage: 7 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [get_response] Total LLM token usage: 1883 tokens\n",
      "> [get_response] Total LLM token usage: 1883 tokens\n",
      "> [get_response] Total LLM token usage: 1883 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [get_response] Total embedding token usage: 0 tokens\n",
      "> [get_response] Total embedding token usage: 0 tokens\n",
      "> [get_response] Total embedding token usage: 0 tokens\n",
      "\u001b[38;5;200m\u001b[1;3m[Tokyo] A: \n",
      "Major festivals in Tokyo that celebrate its culture include the Sannō at Hie Shrine, the Sanja at Asakusa Shrine, and the biennial Kanda Festivals. The last features a parade with elaborately decorated floats and thousands of people. Annually on the last Saturday of July, an enormous fireworks display over the Sumida River attracts over a million viewers. Once cherry blossoms bloom in spring, many residents gather in Ueno Park, Inokashira Park, and the Shinjuku Gyoen National Garden for picnics under the blossoms. Harajuku, a neighborhood in Shibuya, is known internationally for its youth style, fashion and cosplay.\n",
      "\u001b[0m\u001b[32;1m\u001b[1;3m[Tokyo] Q: What events in Tokyo celebrate its art\n",
      "\u001b[0mINFO:llama_index.token_counter.token_counter:> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [retrieve] Total embedding token usage: 7 tokens\n",
      "> [retrieve] Total embedding token usage: 7 tokens\n",
      "> [retrieve] Total embedding token usage: 7 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [get_response] Total LLM token usage: 1188 tokens\n",
      "> [get_response] Total LLM token usage: 1188 tokens\n",
      "> [get_response] Total LLM token usage: 1188 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [get_response] Total embedding token usage: 0 tokens\n",
      "> [get_response] Total embedding token usage: 0 tokens\n",
      "> [get_response] Total embedding token usage: 0 tokens\n",
      "\u001b[32;1m\u001b[1;3m[Tokyo] A: \n",
      "Events in Tokyo that celebrate its art include the Sannō at Hie Shrine, the Sanja at Asakusa Shrine, the biennial Kanda Festivals, the New National Theater Tokyo in Shibuya, the National Noh Theatre for noh, the Kabuki-za for Kabuki, symphony orchestras and other musical organizations, modern Japanese and international pop and rock music venues, and the annual fireworks display over the Sumida River.\n",
      "\u001b[0mINFO:llama_index.token_counter.token_counter:> [get_response] Total LLM token usage: 696 tokens\n",
      "> [get_response] Total LLM token usage: 696 tokens\n",
      "> [get_response] Total LLM token usage: 696 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [get_response] Total embedding token usage: 0 tokens\n",
      "> [get_response] Total embedding token usage: 0 tokens\n",
      "> [get_response] Total embedding token usage: 0 tokens\n",
      "\u001b[38;5;200m\u001b[1;3mquery engine response: \n",
      "Some popular festivals and events in Tokyo that celebrate its culture and art include the Sannō at Hie Shrine, the Sanja at Asakusa Shrine, the biennial Kanda Festivals, picnics under the cherry blossoms in Ueno Park, Inokashira Park, and the Shinjuku Gyoen National Garden, the Harajuku fashion and cosplay events, the New National Theater Tokyo in Shibuya, the National Noh Theatre for noh, the Kabuki-za for Kabuki, symphony orchestras and other musical organizations, modern Japanese and international pop and rock music venues, and the annual fireworks display over the Sumida River.\n",
      "\u001b[0mINFO:llama_index.query_engine.sql_join_query_engine:> query engine response: \n",
      "Some popular festivals and events in Tokyo that celebrate its culture and art include the Sannō at Hie Shrine, the Sanja at Asakusa Shrine, the biennial Kanda Festivals, picnics under the cherry blossoms in Ueno Park, Inokashira Park, and the Shinjuku Gyoen National Garden, the Harajuku fashion and cosplay events, the New National Theater Tokyo in Shibuya, the National Noh Theatre for noh, the Kabuki-za for Kabuki, symphony orchestras and other musical organizations, modern Japanese and international pop and rock music venues, and the annual fireworks display over the Sumida River.\n",
      "> query engine response: \n",
      "Some popular festivals and events in Tokyo that celebrate its culture and art include the Sannō at Hie Shrine, the Sanja at Asakusa Shrine, the biennial Kanda Festivals, picnics under the cherry blossoms in Ueno Park, Inokashira Park, and the Shinjuku Gyoen National Garden, the Harajuku fashion and cosplay events, the New National Theater Tokyo in Shibuya, the National Noh Theatre for noh, the Kabuki-za for Kabuki, symphony orchestras and other musical organizations, modern Japanese and international pop and rock music venues, and the annual fireworks display over the Sumida River.\n",
      "> query engine response: \n",
      "Some popular festivals and events in Tokyo that celebrate its culture and art include the Sannō at Hie Shrine, the Sanja at Asakusa Shrine, the biennial Kanda Festivals, picnics under the cherry blossoms in Ueno Park, Inokashira Park, and the Shinjuku Gyoen National Garden, the Harajuku fashion and cosplay events, the New National Theater Tokyo in Shibuya, the National Noh Theatre for noh, the Kabuki-za for Kabuki, symphony orchestras and other musical organizations, modern Japanese and international pop and rock music venues, and the annual fireworks display over the Sumida River.\n",
      "\u001b[32;1m\u001b[1;3mFinal response: Tokyo, the city with the highest population of 13.96 million people, boasts a vibrant and rich culture with a wide variety of art forms. The city offers traditional Japanese art such as calligraphy and woodblock prints, as well as modern art galleries and museums. Popular festivals and events in Tokyo that celebrate its culture and art include the Sannō at Hie Shrine, the Sanja at Asakusa Shrine, the biennial Kanda Festivals, picnics under the cherry blossoms in Ueno Park, Inokashira Park, and the Shinjuku Gyoen National Garden. Additionally, the city hosts Harajuku fashion and cosplay events, performances at the New National Theater Tokyo in Shibuya, the National Noh Theatre for noh, the Kabuki-za for Kabuki, symphony orchestras, and other musical organizations. Tokyo also has modern Japanese and international pop and rock music venues, as well as an annual fireworks display over the Sumida River.\n",
      "\u001b[0m"
     ]
    }
   ],
   "source": [
    "response = query_engine.query(\n",
    "    \"Tell me about the arts and culture of the city with the highest population\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tokyo, the city with the highest population of 13.96 million people, boasts a vibrant arts and culture scene. It offers a diverse range of art forms, from traditional Japanese art such as calligraphy and woodblock prints to modern art galleries and museums. Some notable art galleries and museums in Tokyo include the Nezu Museum in Aoyama, the National Diet Library, National Archives, and the National Museum of Modern Art near the Imperial Palace. The city also hosts various festivals and events throughout the year that celebrate its culture and art, such as the Sannō at Hie Shrine, the Sanja at Asakusa Shrine, and the biennial Kanda Festivals.\n"
     ]
    }
   ],
   "source": [
    "print(str(response))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[36;1m\u001b[1;3mQuerying SQL database: Useful for translating a natural language query into a SQL query over a table containing city_stats, containing the population/country of each city\n",
      "\u001b[0mINFO:llama_index.query_engine.sql_join_query_engine:> Querying SQL database: Useful for translating a natural language query into a SQL query over a table containing city_stats, containing the population/country of each city\n",
      "> Querying SQL database: Useful for translating a natural language query into a SQL query over a table containing city_stats, containing the population/country of each city\n",
      "> Querying SQL database: Useful for translating a natural language query into a SQL query over a table containing city_stats, containing the population/country of each city\n",
      "INFO:llama_index.indices.struct_store.sql_query:> Table desc str: Schema of table city_stats:\n",
      "Table 'city_stats' has columns: city_name (VARCHAR(16)), population (INTEGER), country (VARCHAR(16)) and foreign keys: .\n",
      "\n",
      "> Table desc str: Schema of table city_stats:\n",
      "Table 'city_stats' has columns: city_name (VARCHAR(16)), population (INTEGER), country (VARCHAR(16)) and foreign keys: .\n",
      "\n",
      "> Table desc str: Schema of table city_stats:\n",
      "Table 'city_stats' has columns: city_name (VARCHAR(16)), population (INTEGER), country (VARCHAR(16)) and foreign keys: .\n",
      "\n",
      "INFO:llama_index.token_counter.token_counter:> [query] Total LLM token usage: 385 tokens\n",
      "> [query] Total LLM token usage: 385 tokens\n",
      "> [query] Total LLM token usage: 385 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [query] Total embedding token usage: 0 tokens\n",
      "> [query] Total embedding token usage: 0 tokens\n",
      "> [query] Total embedding token usage: 0 tokens\n",
      "\u001b[33;1m\u001b[1;3mSQL query: SELECT city_name, population, country FROM city_stats WHERE city_name IN ('Berlin', 'Toronto');\n",
      "\u001b[0m\u001b[33;1m\u001b[1;3mSQL response:  Berlin and Toronto are both major cities with large populations. Berlin has a population of 3.6 million people and is located in Germany, while Toronto has a population of 2.9 million people and is located in Canada.\n",
      "\u001b[0m\u001b[36;1m\u001b[1;3mTransformed query given SQL response: What are the age distributions and ethnic compositions of Berlin and Toronto?\n",
      "\u001b[0mINFO:llama_index.query_engine.sql_join_query_engine:> Transformed query given SQL response: What are the age distributions and ethnic compositions of Berlin and Toronto?\n",
      "> Transformed query given SQL response: What are the age distributions and ethnic compositions of Berlin and Toronto?\n",
      "> Transformed query given SQL response: What are the age distributions and ethnic compositions of Berlin and Toronto?\n",
      "Generated 4 sub questions.\n",
      "\u001b[36;1m\u001b[1;3m[Berlin] Q: What is the age distribution of Berlin\n",
      "\u001b[0mINFO:llama_index.token_counter.token_counter:> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [retrieve] Total embedding token usage: 7 tokens\n",
      "> [retrieve] Total embedding token usage: 7 tokens\n",
      "> [retrieve] Total embedding token usage: 7 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [get_response] Total LLM token usage: 1784 tokens\n",
      "> [get_response] Total LLM token usage: 1784 tokens\n",
      "> [get_response] Total LLM token usage: 1784 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [get_response] Total embedding token usage: 0 tokens\n",
      "> [get_response] Total embedding token usage: 0 tokens\n",
      "> [get_response] Total embedding token usage: 0 tokens\n",
      "\u001b[36;1m\u001b[1;3m[Berlin] A: \n",
      "It is not possible to answer this question with the given context information.\n",
      "\u001b[0m\u001b[33;1m\u001b[1;3m[Berlin] Q: What is the ethnic composition of Berlin\n",
      "\u001b[0mINFO:llama_index.token_counter.token_counter:> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [retrieve] Total embedding token usage: 7 tokens\n",
      "> [retrieve] Total embedding token usage: 7 tokens\n",
      "> [retrieve] Total embedding token usage: 7 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [get_response] Total LLM token usage: 1817 tokens\n",
      "> [get_response] Total LLM token usage: 1817 tokens\n",
      "> [get_response] Total LLM token usage: 1817 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [get_response] Total embedding token usage: 0 tokens\n",
      "> [get_response] Total embedding token usage: 0 tokens\n",
      "> [get_response] Total embedding token usage: 0 tokens\n",
      "\u001b[33;1m\u001b[1;3m[Berlin] A: \n",
      "The ethnic composition of Berlin is diverse. According to the 2019 Berlin residents register, the population of Berlin is composed of approximately 79.2% German citizens, 11.6% with a migration background from one of the Member states of the Organization of Islamic Cooperation, 4.2% with a migration background from other countries, and 5% with no migration background.\n",
      "\u001b[0m\u001b[38;5;200m\u001b[1;3m[Toronto] Q: What is the age distribution of Toronto\n",
      "\u001b[0mINFO:llama_index.token_counter.token_counter:> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [retrieve] Total embedding token usage: 7 tokens\n",
      "> [retrieve] Total embedding token usage: 7 tokens\n",
      "> [retrieve] Total embedding token usage: 7 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [get_response] Total LLM token usage: 1891 tokens\n",
      "> [get_response] Total LLM token usage: 1891 tokens\n",
      "> [get_response] Total LLM token usage: 1891 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [get_response] Total embedding token usage: 0 tokens\n",
      "> [get_response] Total embedding token usage: 0 tokens\n",
      "> [get_response] Total embedding token usage: 0 tokens\n",
      "\u001b[38;5;200m\u001b[1;3m[Toronto] A: \n",
      "In 2016, persons aged 14 years and under made up 14.5 per cent of the population, and those aged 65 years and over made up 15.6 per cent. The median age was 39.3 years. The city's gender population is 48 per cent male and 52 per cent female. Women outnumber men in all age groups 15 and older.\n",
      "\u001b[0m\u001b[32;1m\u001b[1;3m[Toronto] Q: What is the ethnic composition of Toronto\n",
      "\u001b[0mINFO:llama_index.token_counter.token_counter:> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [retrieve] Total embedding token usage: 7 tokens\n",
      "> [retrieve] Total embedding token usage: 7 tokens\n",
      "> [retrieve] Total embedding token usage: 7 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [get_response] Total LLM token usage: 1962 tokens\n",
      "> [get_response] Total LLM token usage: 1962 tokens\n",
      "> [get_response] Total LLM token usage: 1962 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [get_response] Total embedding token usage: 0 tokens\n",
      "> [get_response] Total embedding token usage: 0 tokens\n",
      "> [get_response] Total embedding token usage: 0 tokens\n",
      "\u001b[32;1m\u001b[1;3m[Toronto] A: \n",
      "The ethnic composition of Toronto is diverse. In 2016, the three most commonly reported ethnic origins overall were Chinese (332,830 or 12.5 per cent), English (331,890 or 12.3 per cent) and Canadian (323,175 or 12.0 per cent). Common regions of ethnic origin were European (47.9 per cent), Asian (including Middle-Eastern – 40.1 per cent), African (5.5 per cent), Latin/Central/South American (4.2 per cent), and North American aboriginal (1.2 per cent). In 2016, 51.5 per cent of the residents of the city proper belonged to a visible minority group, with the largest visible minority groups being South Asian (Indian, Pakistani, Sri Lankan at 338,960 or 12.6 per cent), East Asian (Chinese at 332,830 or 12.5 per cent), and Black (239,850 or 8.9 per cent).\n",
      "\u001b[0mINFO:llama_index.token_counter.token_counter:> [get_response] Total LLM token usage: 746 tokens\n",
      "> [get_response] Total LLM token usage: 746 tokens\n",
      "> [get_response] Total LLM token usage: 746 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [get_response] Total embedding token usage: 0 tokens\n",
      "> [get_response] Total embedding token usage: 0 tokens\n",
      "> [get_response] Total embedding token usage: 0 tokens\n",
      "\u001b[38;5;200m\u001b[1;3mquery engine response: \n",
      "The age distribution of Berlin is not known with the given context information. The ethnic composition of Berlin is 79.2% German citizens, 11.6% with a migration background from one of the Member states of the Organization of Islamic Cooperation, 4.2% with a migration background from other countries, and 5% with no migration background.\n",
      "\n",
      "The age distribution of Toronto is 14.5 per cent of the population aged 14 years and under, 15.6 per cent aged 65 years and over, and a median age of 39.3 years. The city's gender population is 48 per cent male and 52 per cent female. The ethnic composition of Toronto is diverse, with the three most commonly reported ethnic origins overall being Chinese (12.5 per cent), English (12.3 per cent) and Canadian (12.0 per cent). Common regions of ethnic origin were European (47.9 per cent), Asian (including Middle-Eastern – 40.1 per cent), African (5.5 per cent), Latin/Central/South American (4.2 per cent), and North American aboriginal (1.2 per cent). In 2016, 51.5 per cent of the residents of the city proper belonged to a visible minority group, with the largest visible minority groups being South Asian (12.6 per cent), East Asian (12.5 per cent), and Black (8.9 per cent).\n",
      "\u001b[0mINFO:llama_index.query_engine.sql_join_query_engine:> query engine response: \n",
      "The age distribution of Berlin is not known with the given context information. The ethnic composition of Berlin is 79.2% German citizens, 11.6% with a migration background from one of the Member states of the Organization of Islamic Cooperation, 4.2% with a migration background from other countries, and 5% with no migration background.\n",
      "\n",
      "The age distribution of Toronto is 14.5 per cent of the population aged 14 years and under, 15.6 per cent aged 65 years and over, and a median age of 39.3 years. The city's gender population is 48 per cent male and 52 per cent female. The ethnic composition of Toronto is diverse, with the three most commonly reported ethnic origins overall being Chinese (12.5 per cent), English (12.3 per cent) and Canadian (12.0 per cent). Common regions of ethnic origin were European (47.9 per cent), Asian (including Middle-Eastern – 40.1 per cent), African (5.5 per cent), Latin/Central/South American (4.2 per cent), and North American aboriginal (1.2 per cent). In 2016, 51.5 per cent of the residents of the city proper belonged to a visible minority group, with the largest visible minority groups being South Asian (12.6 per cent), East Asian (12.5 per cent), and Black (8.9 per cent).\n",
      "> query engine response: \n",
      "The age distribution of Berlin is not known with the given context information. The ethnic composition of Berlin is 79.2% German citizens, 11.6% with a migration background from one of the Member states of the Organization of Islamic Cooperation, 4.2% with a migration background from other countries, and 5% with no migration background.\n",
      "\n",
      "The age distribution of Toronto is 14.5 per cent of the population aged 14 years and under, 15.6 per cent aged 65 years and over, and a median age of 39.3 years. The city's gender population is 48 per cent male and 52 per cent female. The ethnic composition of Toronto is diverse, with the three most commonly reported ethnic origins overall being Chinese (12.5 per cent), English (12.3 per cent) and Canadian (12.0 per cent). Common regions of ethnic origin were European (47.9 per cent), Asian (including Middle-Eastern – 40.1 per cent), African (5.5 per cent), Latin/Central/South American (4.2 per cent), and North American aboriginal (1.2 per cent). In 2016, 51.5 per cent of the residents of the city proper belonged to a visible minority group, with the largest visible minority groups being South Asian (12.6 per cent), East Asian (12.5 per cent), and Black (8.9 per cent).\n",
      "> query engine response: \n",
      "The age distribution of Berlin is not known with the given context information. The ethnic composition of Berlin is 79.2% German citizens, 11.6% with a migration background from one of the Member states of the Organization of Islamic Cooperation, 4.2% with a migration background from other countries, and 5% with no migration background.\n",
      "\n",
      "The age distribution of Toronto is 14.5 per cent of the population aged 14 years and under, 15.6 per cent aged 65 years and over, and a median age of 39.3 years. The city's gender population is 48 per cent male and 52 per cent female. The ethnic composition of Toronto is diverse, with the three most commonly reported ethnic origins overall being Chinese (12.5 per cent), English (12.3 per cent) and Canadian (12.0 per cent). Common regions of ethnic origin were European (47.9 per cent), Asian (including Middle-Eastern – 40.1 per cent), African (5.5 per cent), Latin/Central/South American (4.2 per cent), and North American aboriginal (1.2 per cent). In 2016, 51.5 per cent of the residents of the city proper belonged to a visible minority group, with the largest visible minority groups being South Asian (12.6 per cent), East Asian (12.5 per cent), and Black (8.9 per cent).\n",
      "\u001b[32;1m\u001b[1;3mFinal response: Berlin and Toronto are both major cities with large populations. Berlin has a population of 3.6 million people and is located in Germany, while Toronto has a population of 2.9 million people and is located in Canada. The ethnic composition of Berlin is predominantly German citizens (79.2%), with 11.6% having a migration background from one of the Member states of the Organization of Islamic Cooperation, 4.2% from other countries, and 5% with no migration background.\n",
      "\n",
      "On the other hand, Toronto has a diverse ethnic composition, with the three most commonly reported ethnic origins being Chinese (12.5%), English (12.3%), and Canadian (12.0%). Common regions of ethnic origin include European (47.9%), Asian (including Middle-Eastern – 40.1%), African (5.5%), Latin/Central/South American (4.2%), and North American aboriginal (1.2%). In 2016, 51.5% of Toronto's residents belonged to a visible minority group, with the largest visible minority groups being South Asian (12.6%), East Asian (12.5%), and Black (8.9%).\n",
      "\n",
      "The age distribution of Toronto is 14.5% of the population aged 14 years and under, 15.6% aged 65 years and over, and a median age of 39.3 years. The city's gender population is 48% male and 52% female. Unfortunately, the age distribution of Berlin is not known with the given context information.\n",
      "\u001b[0m"
     ]
    }
   ],
   "source": [
    "response = query_engine.query(\n",
    "    \"Compare and contrast the demographics of Berlin and Toronto\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Berlin's history dates back to the early 13th century when it was founded as a small settlement. In 1618, the Margraviate of Brandenburg entered into a personal union with the Duchy of Prussia, and in 1701, they formed the Kingdom of Prussia with Berlin as its capital. The city grew and merged with neighboring cities, becoming a center of the Enlightenment under the rule of Frederick the Great in the 18th century.\n",
      "\n",
      "The Industrial Revolution in the 19th century transformed Berlin, expanding its economy, population, and infrastructure. In 1871, it became the capital of the newly founded German Empire. The early 20th century saw Berlin as a hub for the German Expressionist movement and a major world capital known for its contributions to science, technology, arts, and other fields.\n",
      "\n",
      "In 1933, Adolf Hitler and the Nazi Party came to power, leading to a decline in Berlin's Jewish community and the city's involvement in World War II. After the war, Berlin was divided into East and West Berlin, with the former under Soviet control and the latter under the control of the United States, United Kingdom, and France. The Berlin Wall was built in 1961, physically and ideologically dividing the city until its fall in 1989. Following the reunification of Germany in 1990, Berlin once again became the capital of a unified Germany and has since continued to grow and develop as a major global city.\n"
     ]
    }
   ],
   "source": [
    "print(str(response))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "llama_index_v2",
   "language": "python",
   "name": "llama_index_v2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
